AI Hallucinations
Alejandro Piad Morffis explains Why reliable AI requires a paradigm shift
Hallucinations are not a bug. They’re an unavoidable consequence, or to put it more bluntly, a feature of the statistical underpinnings that make these models efficient.
Sometimes hallucinations are caused by bad training data. Technically this isn’t an hallucination.
In more technical terms, the statistical model assumes a smooth distribution, which is necessary because the size of the data the model needs to encode is orders of magnitude larger than the memory (i.e., the number of parameters) in the model. Thus, the models must compress the training corpus, and compression implies losing some information.
Recent research suggests that if there is a sentence that can be generated at all, no matter how low its base probability, then there is a prompt that will generate it with almost 100% certainty.
by Nov 2024, even Noah Smith is starting to suspect
It’s seeming more likely that hallucination is not a form of variance — a statistical error that can be eliminated with larger samples — but a form of asymptotic bias that will always plague LLMs no matter how big they get. LLMs seem unable to functionally distinguish truth from falsehood.
Gerben Wierda is “Trusted Advisor in Strategic/Complex IT. Author, speaker on the same, and on IT and society in general.” and says “we should refrain from calling these errors or hallucinations. We might call them ‘failed approximations’ to signal the correct ones are also ‘approximations’.”
Mar 31·edited Mar 31_Liked by Gary Marcus with an interesting thread of comments discussing the claim:
What still isn’t clear to most people is that with GenAI
useful memorization and unacceptable training data leakage are technically the same thing
creativity and ‘hallucinations’ are technically the same thing
They are the same. We just stick two different labels on them based on if we like/want the result or not |
Measuring Hallucination
